Most activities in a multimedia system tend to be of the real-time variety. Put simply, a real time system such as a video server is one in which, a datastream must be written to storage and accessed without delay or interruption. Such delays give rise to lost frames or dropout in program content which is unacceptable in these applications.
In other multimedia applications it may be possible to provide some delay to effect storage of data which is then played back at a later time. However, it should be readily apparent that again taking the video server or video-on-demand type of applications as examples, it would be completely unacceptable for viewers to experience blank spaces during their viewing of a favorite movie while the system is catching up with writes to disk.
Real-time applications have been known in the art for some time, such as in process control. However these systems may be distinguished over those in which the invention is particularly applicable in the sense that although they may involve real-time data, it is oftentimes permissible to miss portions of data for varying reasons. For example, in a process control situation, when data packets are missed, they may simply be transmitted at a later time in response to various error checking mechanisms. Alternatively, in such systems, it may be acceptable to simply miss frames of data in their entirety. Parameters may be measured on a cyclical basis at such a rate that any one frame of data may essentially be redundant. In other words another measurement will occur in a short enough period of time after the data dropout so as to cause the dropout to be insignificant.
These more forgiving aspects of other real-time applications simply are not possible in the multimedia real-time systems under consideration at present, as in the case of video-on-demand where the customers simply find it intolerable to miss any of the video datastream content. Examples of streaming data in such applications might include video title content being received from an archive server which is written into some disk array or other storage device and is being read out immediately to a plurality of clients. Similarly, a real-time press-conference might be transpiring, a sports event, or other datastream which may be incoming for example from a satellite link in real-time. In such instances it is essential that this data be recorded or written to a file essentially as it is incoming as a datastream, and moreover to provide for its readout to the clients at essentially the same rate without delay.
One problem in handling the latter type of real-time systems is precisely because there is simply no luxury of time to be able to temporarily halt the recording process and resume it later because of the data loss as a result of this to the end-user. Put simply, it is completely unacceptable to block the recording process due to resource allocation problems or the like for even a slight period of time.
This real-time write feature relates to a broader concept known in the field as a quality of service (QOS) constraint that multimedia file systems must adhere to in real-time applications. The need for such a QOS is extremely pronounced in multimedia-based system solutions as just explained, wherein the applications simply must have the ability to load and record multimedia content in real-time. One nuance of this problem is that multimedia files tend to be extremely large in size. Thus a need has arisen to provide system designs which provide for real-time writes which will never block while waiting for system resources (e.g., disk bandwidth, buffer allocation, and disk map allocation) irrespective of the size or duration of the incoming record.
Several approaches have attempted to address this problem, all of which are essentially predicated on some manner of intelligent estimating of the characteristics of the incoming datastream. Two characteristics of particular importance are the total expected size of the file to be written and the datastream rate, both of which are problematical to estimate for obvious reasons. For example in a real-time videoconference it may be difficult to estimate how long the videoconference will last. As to the datastream rate, one might have some idea as to what the incoming rate is but with little assurance, the guess is correct. Moreover, the rate may vary due to differing compression algorithms and the like.
Notwithstanding these limitations, one approach is to estimate as best one could how large the complete incoming file would be and the datastream rate, and then simply to allocate resources based upon this predetermined estimate through manual input.
A serious problem with this approach is that if one overestimates the required resources to handle the entire file which are then committed in advance to the datastream, this may compromise the quality of service of other applications whose requirements may be equally as demanding and involving perhaps hundreds or thousands of clients. If, on the other hand, needed resources based on these unreliable estimates are underestimated, the very real and serious risk of the particular application in question suffering frame dropouts and associated deterioration in QOS occurs, given that the real-time datastream might have to be interrupted until adequate resource is available.
Additionally, even if the estimate is right, merely dedicating statically a required resource allocation is oftentimes undesirable. Although the file size may have been accurately known a priori prior to transmission, events may occur during the transmission which alter this otherwise accurate estimate. This thereby would render any such static commitment of resource unwise and detrimental to optimum system performance and the ability to provide a balanced system so that no one application is penalized.